feat(compile): add pool-overrides for per-job pool configuration (#1464)#1472
Conversation
Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
…ed signature Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
|
/rust-review |
|
✅ Rust PR Reviewer completed successfully! |
🔍 Rust PR ReviewSummary: Looks good — the implementation is correct and well-tested. One fragility concern and one minor nit worth noting. Findings
|
|
@copilot Fix up PR description and implement feedback and suggestions. |
… EMPTY_OVERRIDES static Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
Done in commit 03505df. Two improvements from the rust review:
Updated PR description (reflecting the SummaryThe compiler previously cloned the top-level pool:
name: SpecializedLinuxPool
overrides:
detection:
vmImage: ubuntu-22.04
safe-outputs:
vmImage: ubuntu-22.04
conclusion:
vmImage: ubuntu-22.04Per-job overrides are nested under
Constraints: Docs: Full reference section added to Test plan12 new tests: 9 unit tests in |
|
/rust-review |
|
✅ Rust PR Reviewer completed successfully! |
🔍 Rust PR ReviewSummary: Looks good overall — clean design, solid test coverage, good error handling. A couple of things worth a second look. Findings🔒 Security Concerns
|
|
@copilot implement suggestions |
Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
Done in the latest commit. Three changes from the second rust-review:
|
|
@copilot why does the new override frontmatter support legacy strings? They are automatically transformed by the codemod before compilation. The new code should never support legacy string format. |
…legacy strings) Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
Fixed in 6f879b6. Changed |
Remove the untagged PoolConfig::Name(String) variant and collapse PoolConfigFull into PoolConfig. Legacy scalar pool: sources are still accepted as input and migrated to object form by codemod 0002_pool_object_form before typed deserialization, so the typed layer is object-only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: fc152016-8608-47c3-bb4b-30d3565e63f0
Production code in resolve_pool_typed read the pub fields directly, which left name()/vm_image() marked #[cfg(test)] and os() marked #[allow(dead_code)]. Use the accessors consistently so they are genuinely production-used and drop the test-only/dead-code attributes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: fc152016-8608-47c3-bb4b-30d3565e63f0
Summary
The compiler previously cloned the top-level
pool:onto every generated job,forcing lightweight downstream jobs (Detection, SafeOutputs, Conclusion) onto
scarce specialized capacity. This adds
pool.overrides:to right-sizeindividual jobs independently.
Per-job overrides are nested under
pool.overrides:(a sub-key ofpool:, nota separate top-level key). Each override value uses the object pool form —
the same schema as the top-level
pool:.Valid override keys:
setup,agent,detection,safe-outputs,safe-outputs-reviewed,teardown,conclusion.safe-outputs-reviewedinherits the
safe-outputsoverride unless it has its own entry.manual-reviewis always rejected (the ManualReview job is agentless and fixed to
pool: server).Unknown keys emit a compiler warning and are ignored (forward-compat).
Changes
src/compile/types.rs—overrides: HashMap<String, PoolConfig>field addedto the pool object. The legacy untagged
PoolConfigenum (scalarName(String)Full) was removed and the object struct (formerlyPoolConfigFull) is now thesingle
PoolConfigtype. The typed layer is object-form only: the legacyscalar shorthand
pool: MyPoolis still accepted as source input and rewrittento
pool: { name: MyPool }by codemod0002_pool_object_formbefore typeddeserialization, so a bare string never reaches the type. Pool accessors
(
name/vm_image/os/demands/overrides) live on the struct and are used byboth production and tests.
src/compile/common.rs—PerJobPoolsstruct (one resolvedPoolpercanonical job) +
resolve_pool_overrides_typed()which applies overrides,inherits
safe-outputs-reviewedfromsafe-outputs, warns on unknown keys, andhard-errors on
manual-review.resolve_pool_typedgoes through thePoolConfigaccessors.
src/compile/agentic_pipeline.rs—StandaloneCtx.pool: Pool→pools: PerJobPools; each of the job builders reads its own field.SafeOutputsVariantcarries an
is_reviewed: boolso the reviewed-variant pool selection no longerrelies on a string comparison against the job base name.
src/compile/onees_ir.rs— compile-time error ifpool.overrides:isnon-empty on
target: 1es; the 1ES template owns pool selection.Constraints: Each override is validated through the same
resolve_pool_typedrules as the default pool (
name:andvmImage:are mutually exclusive;demands:requiresname:). Override values accept only the object form — a barepool-name string is not accepted (use
name:).Docs: Full reference section added to
front-matter.md(key table + trustboundary note covering
agent/setupsource-tree exposure and Detection/SafeOutputs/Conclusion
SC_WRITE_TOKENexposure); trust boundary callout added tosafe-outputs.md.Test plan
Unit tests in
common.rscover default inheritance, per-key overrides,safe-outputs-reviewedinheritance,manual-reviewrejection, invalid poolcombos, and demands composition; integration tests in
agentic_pipeline.rsverifypools flow through
build_canonical_jobsto the emitted job objects; aparse_markdownregression test confirms a legacy scalarpool:source stillcompiles via the codemod.
cargo testandcargo clippy --all-targets --all-featurespass clean.